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Abstract 
The  strong  consistency  of  regression  quantile  statistics  (Koenker  and 
Bassett  (1978))  in  linear  models  with  iid  errors  is  established.  Mild  regularity 
conditions  on  the  regression  design  sequence  and  the  error  distribution  are 
required.  Strong  consistency  of  the  associated  empirical  quantile  process 
(introduced  in  Bassett  and  Koenker  (1982))  is  also  established  under  analo- 
gous conditions.  However,  for  the  proposed  estimate  of  the  conditional  distri- 
bution function  of  Y,  no  regularity  conditions  on  the  error  distribution  are 
required  for  uniform  strong  convergence,  thus  establishing  a  Glivenko- 
Cantelli-type  theorem  for  this  estimator. 
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1.   Introduction 

In  several  recent  papers,  Koenker  and  Bassett  (1978,  1982)  and  Bassett 
and  Koenker  (1982),  we  have  explored  the  problem  of  estimating  linear 
models  for  conditional  quantile  functions  of  related  random  variables.  This 
approach  complements  classical  least-squares  methods  for  linear  models  as 
well  as  recent  robust  methods  which  focus  exclusively  on  estimation  of  condi- 
tional central  tendency. 

th 
We   might   hypothesize   that   the   0      conditional   quantile   function   of 

Y  ,  ...,  Y    is  a  linear  function  of  a  p- vector  of  exogenous  variables  x  ,  i.e., 

QY(8   |  x)=*0,  (1.1) 

Given  this  hypothesis,  one  may  ask  under  what  conditions  can  we  con- 
sistently estimate  the  parameter  vector  |3  ?  In  Koenker  and  Bassett  (1978) 
we  showed  that  for  the  linear  model  with  iid  errors  any  sequence  of  solutions 
{3fl}  t°  the  problem 


min    E  p9(y,-  -*,-*)  (i.2) 

b  €  R'    .     , 

»  =1 

where  p(-)  is  the  check  function. 


Pa(«)   = 


'    e«        u  s=  o 
ie  - 1)«     u.  <o 

had  the  property  that,  under  mild  conditions  on  the  sequence  of  designs  and 
the  assumption  that  F  had  a  positive  density  in  a  neighborhood  of 
Q(9)  =  F    (8),    vn   (ft    _  f$)    converged    in    law   to   a   /:-variate   Gaussian 


distribution  with  mean  vector  £  =  (Q  (8),  0,  ...,  0)'  €  R  .  Thus,  under  the 
foregoing  conditions  pfl  is  weakly  consistent  for  Bfl  =  B  +  £  In  Koenker 
and  Bassett  (1982)  we  showed  that  similar  asymptotic  behavior  prevailed  in 
sequences  of  linear  models  with  heteroscedasticity  of  order  0(I/Vn  ). 

Here  we  maintain  the  hypothesis  of  iid  errors  while  relaxing  our  previ- 
ous smoothness  conditions  on  the  error  distribution  F .  We  begin  by  treat- 
ing the  behavior  of  the  p  -dimensional  regression  quantiles  and  conclude  by 
treating  the  associated  empirical  processes  introduced  in  Bassett  and 
Koenker  (1982).  Almost  sure  convergence  results  are  established  under  mild 
regularity  conditions  on  the  design  and  the  distribution  function  of  the 
errors.  In  the  case  of  our  proposed  estimate  of  the  error  distribution  no 
regularity  conditions  are  required  on  F ,  thus  providing  a  natural  extension 
of  the  Glivenko-Cantelli  theorem  to  the  realm  of  linear  models.  The  latter 
result  provides  an  intriguing  alternative  to  methods  based  on  residuals  for 
assessing  distributional  features  of  linear  models  with  iid  errors.  Applica- 
tions to  bootstrapping  and  other  model  diagnostics  immediately  suggest 
themselves. 

2.    Strong  Consistency  of  Regression  Quantiles 

We  will  assume  throughout  that  we  have  data  generated  from  the 
model, 


Y.  =  x.  8  +  u. ,  (2.1) 

The  errors  u.    are  assumed  to  be  independently  and  identically  distributed 

with  distribution  function  F.    About  F   we  will  assume  that  it  is  a  proper, 

right-continuous  distribution  function.   Its  "inverse"  will  be  denoted  by 

Q(0)  =inf{u   \F(u)^*}  (2.2) 

so  Q   is  left  continuous  on  [0,  lj.    The  parameter  (3  is  an  unknown  p  -vector. 

The  sequence  {x.  }  of  fixed  design  vectors  is  assumed  to  "contain  an  inter- 
cept," that  is,  x.  =  1  for  all  i  and  to  satisfy  the  following  regularity  condi- 
tions: 

lim  inf    d     =  lim  inf       inf         n      E  \  x.  <u  |  =  d  >  0  (Ti\\ 

n  -*  ]    I  (i)  |    |    =  1 

-1  2 

lim  inf   D     =  lim  inf      sup        n      E(x.  a>)    =D    <  <»  (T)^ 

71    -*  |   (I)    |      |      =    1 

We  may  now  state: 

Theorem  1.    If  Dl  and  D2  hold  and  F   has  a  unique  8     quantile  then 
any  sequence  of  solutions  {p    (0)}  to  problem  (1.3)  satisfies  3    (0)  -» (3  +  (L 
almost  surely. 
Proof.    FLx  8  and  consider, 

n 

*,  (8)  -  n  -1   £  r.  (8)  (23) 

t   =1 


n 
-1 


=  n       £  [p9(a.   -  x.  8  -  Q  (8))  -  p,(«.  -  Q  (6))] 

t'  =1 

Since  R    (0)  =  0,  and  R    (8)  is  a  sum  of  convex  functions  and  therefore  con- 
vex, it  suffices  to  show  that  for  any  A  >  0 


lim  inf         inf         R   (8)  >  0     a.s. 

n    -x       I    18  I    I    =A 


(2.4) 


We  begin  by  establishing  that 


Rn  (8)  -  E  Rn  (8)     a.s.  (2.5) 

Since  r.  (8)  <   |  j.  8  |  ,  Kolmogorov's  criterion  yields 


o        o 


J7  Var(r.  (8))/t    *  £(*.  8)  /•  (2.6) 


t    =1 


s  |  |s|  I'p'rti.sr/trfx.sn2 

which  is  convergent  so  (2.5)  follows.    This  may  be  strengthened  to  uniform 
convergence  on  compacts  by  noting  that  for  any  8    €  R    ,  we  have. 

sup  |  R   (8)  -  R   (8Q)  |   <  sup{n  AE  \  x.  (8  -  8Q)  | } 


s  -  60  :    s  « 


(2.7) 


€         sup  n      E  J  X.  0)  I 

I  leal  I  -1 


n 


1/2 

So   it   remains   to   show   that   ER    (8)   is   bounded    away   from   0,   for   any 


8  I   I    >  0.   For  a  >  0, 

g(a)=E  [pe(ti    -  a  -  Q  (6))  -  p9(U    -  Q  (6)]  (2.8) 

Q   +  a  *  Q   +  a 

=     /      (1  -e)a<ZF(u)  -     /      a6rfF(it)-     /    (u-Q)dF(u) 

-a  g  +  a  Q 

and  integrating  by  parts  gives, 

Q  +  a 

<7(a)  =     /      (F(u)  -Q)du    .  (2.9) 

Q 

For  a  <  0,  the  sign  and  limits  of  integration  are  simply  reversed.  The  func- 
tion g  is  convex,  g  (0)  =0,  and  g  (a)  >  0  for  any  a  #  0  by  the  uniqueness 
of  the  6  quantile.  Now,  let  h  (a)  denote  the  convex  hull  of  g  (a)  and  g  (-a) 
then  we  have, 

n 

^n(8)>n_1    £  *  (*,■»)  (2.10) 


>  n~lEh(  \x.  8  |) 


-i 


h{n~  E  |x.  8  |) 


>A(<*  |  |8|  |) 
by   the   symmetry   of   A  ,    Jensen's    inequality    and    (D2)    respectively.     The 


function  h  (a)  is  bounded  away  from  zero  for  a  £  0  by  the  uniqueness  of  the 
8     quantile,  thus  completing  the  proof. 

Reviewing  the  preceding  argument  it  is  clear  that  uniqueness  of  the  8 
quantile  is  needed  only  to  argue  that  ER  (8)  has  a  unique  minimum  at  the 
origin.  When  the  8  quantile  is  not  unique  then  ER  (8)  has  a  larger 
minimum  set,  but  it  is  straightforward  to  show  that  solutions  to  (1.3)  con- 
verge almost  surely  to  elements  of  this  set.  See  Koenker  and  Bassett(1984) 
for  an  example  of  weak  but  not  strong  convergence  in  this  framework.  The 
following  result  will  be  used  in  subsequent  sections. 

Theorem  2.2.  Suppose  F  is  any  proper,  right  continuous  distribution 
function  and  D1-D2  prevail.  Let  An  (A)  =  {5  €  RP  \ERn  (8)  ^  A},  the  A~ 
level  set  of  ER  (8).  Then  any  sequence  of  solutions  to  (1.3),  {p  (6)}, 
satisfies 

Pft(e)-p-S9€.dn(4)    a.s.  (2.11) 

for  all  A  >  0. 

Proof.    When   Q  (8  +  0)  =  Q  (8)  this  follows  immediately  from  Theorem  1 
since  trivially,  0  €  A    (A)  for  all  A.   When, 

£(8)  =  Q(8  +  0)  -^(8)  >0  (2.12) 

the  function  g  (a)  is  identically  zero  on  [0,  £(0)]  and  therefore  ER    (8)  will 

attain  a  minimum  of  zero  on  a  set  containing  the  origin.   It  suffices  to  verify 

that  the  (necessarily  convex)  sets  A    (A)  are  bounded  since  we  may  then,  by 


the  prior  argument,  finitely  cover  the  boundary  of  any  such  set  and  by  the 
uniform  convergence  of  R  (8)  to  ER  (3)  on  compacts  we  may  conclude  that 
any  solution  to  (1.3)  lies  inside  this  boundary. 

For  any  A.  €  A  (A)  we  have  h(d  \\\  |)  ^  A  by  the  argument  of 
(2.10).  Since  h(-)  is  convex  and  zero  only  on  the  bounded  interval 
[-£(9),  £(6)]>  f°r  an>'  -1  >  0,  there  exists  an  M  <  »  such  that  h  (a)  ^  A 
implies  |a|  <  A/ .  Thus  |  |X|  |  <  M /d  for  any  X.  €  A  (A),  and  this 
completes  the  proof. 

3.    Strong  Convergence  of  Empirical  Processes  Based  on  Regression  Quantiles 

In  Bassett  and  Koenker  (1982)  we  proposed  an  estimate  of  the  condi- 
tional quantile  function  of  Y  given  x  , 

QY(B\x)  =  m!{xb   |  6    <E  5(6)}  (3.1) 

where  as  above 

B  (9)  =  {6    €  RP  |  27  pf(f .   -  x.  6  )  =  min!}  .  (3.2) 

There  it  is  shown  that  at  x  =  x  =  n  Ex.,  the  sample  paths  of  the  ran- 
dom function 

Qr(e)  =  Q'r(e|7)  (3.3) 

are  non-decreasing,  left  continuous  jump  functions  on  (0,  1).  However, 
unlike  the  ordinary  empirical  quantile  function  to  which  QYW  specializes 
when  X  =  1  ;  QY(Q),  jumps  at  irregularly  spaced  points  on  (0,  1).  Simi- 
larly, one  may  show  that 


Qy(e  +  0)»  lim  gy(Q  +  €)  (3.4) 

=  sup{B   \b    €  5(6)} 

is  a  nondecreasing,  right-  continuous  jump  function  on  (0,  1).  It  was  also 
shown  that  properly  normalized  versions  of  these  processes  have  finite 
dimensional  distributions  which  converge  to  those  of  the  Brownian  bridge 
process.  Portnoy  (1983)  has  recently  strengthened  these  results  to  establish 
weak  convergence  of  the  estimate 

FY(y)=sup{Q\QY(Q)^y}  (3.5) 

to  the  Brownian  bridge. 

Here  we  wish  to  investigate  the  strong  convergence  of  QY{Q)  and  Fy(Q) 
using  results  from  the  previous  section.    We  may  begin  by  noting  that  given 

th 

the  iid  error  assumption  of  model  (1.2),  the  6  conditional  quantile  function 
of  Y  given  x  may  be  written  as, 

QY(9\z)=zl+Q(9)  (3.6) 

We  will  restrict  attention  as  previously  to 

<9r(e)  =  gr(e|j)  =  pj  +  Q(e).  (3.7) 

We  may  now  state: 

Theorem  3.1.    If  Dl  and  D2  hold  and   Q  (0)  is  continuous  on  a  closed 
interval  0C(O,  1)  then 


9 

sup    I    QY(Q)  -QY(Q)\  -0     a.s.  (38) 

e  €  e 

Proof.    From  Theorem  2.1  we  have  pointwise  convergence  of   QY(Q)  and 

Q    (9  +  0)  to  QY(Q)  and  using  the  monotonicity  and  continuity  of  QY(Q) 

this  may  be  strengthened  to  uniform  almost  sure  convergence  on  0.   See,  for 

example,  the  argument  in  Billingsley  (1979,  p.  233)  for  the  Glivenko-Cantelli 

theorem. 

When  there  are  jumps  in  QY(')  some  new  difficulties  arise,  since  QY(Q) 
may  oscillate  between  Qy(6)  and  QY(Q  +  0),  but  based  on  Theorem  2.2  we 
may  establish  the  following  result. 

Theorem  3.2.    If  Dl  and  D2  hold  then 


Y\u  )  -ry[u  )  |   -u     a.s.  ^  gj 

«    6  R 


sup     \FY(u)  -FY{u)\  -Q    a.s 

u  6  R 
Proof.    We  will  consider  the  case  of  continuous  F  first;  discontinuities  in  F 

are  treated  in  the  Appendix.    We  will  begin  by  fixing  8  and  establishing  the 

inequalities: 

liminf  Qy(Q)  >  Qy(Q)      a.s.  (3  10) 

n    -  * 

lim  sup  QY{Q  +  0)  <  Qy(Q  +  0)      a.s.  ^  nj 

From  Theorem  2.2  we  have  for  anv 

QY(0)  ^  QY(Q)  +  inf  {XI  |  X  £  ,ln  (^)}  (3.12) 

and 


10 


QY(Q  +  0)  ^  QY(B)  +  sup  {\ J  |  X  €  An  (A)}  (3.13) 

so,  by  Jensen's  inequality,  using  Dl, 


n 
-1 


{\x    |  X  €  An  (A)}  =  {\x  |  n  '   £  g  (x.  \)  <  A}  (3  14) 

»  =1 

C{\x  \g{\x)  <  A}  . 

Since  g  (•)  is  convex  and  zero  only  on  [0,  Q  (0  +  0)  -  Q  (8)]  we  have 

-o  (A)  ^  inf  {a  |  g  (a)  <  4}  (3.15) 

<  sup  {a  |  £  (a)  <  4} 

<;  Q(8  +  0)  -Q(Q)  +  o(zi) 
so  the  inequalities  follow  letting  A  -  0.    It  is  straightforward  to  verify  that 

(3.10-11)  imply 

lim  sup  FY  ( Qy  (6))  ^  Fy  ( gy  (6))  (3  16) 

and 

lim  inf  Fy(<?y(e)  -0)  a/(<?r(6)  -0)  .  (3  17) 

So  by  continuity  of  F  we  have 


lim  inf  FY (QY(Q)-0)=  lim  sup  Fy ( £y (6))  =  6  .  (3  lg) 

And  by  standard  arguments  for  the  Glivenko-Cantelli  theorem,  again  see, 
Billingsley  (1979,  p.  233),  this  implies  (3.9). 


11 


Appendix 


When  F  has  discontinuities  some  new  difficulties  arise  which  are  taken 
up  in  this  appendix.  Principally,  we  must  verify  that  F  converges  to  a 
proper  distribution  function.  See  the  remark  following  the  the  proof  of  the 
lemma.   To  complete  the  proof  of  Theorem  3.2  we  will  need: 

Lemma.    Fix  0€(O,1),  and  assume,  £(Q)=F  [Q  (Q))-F  (Q  (8)-0)  >  0,  so 

th 
F  has  strictly  positive  mass  at  the  0     quantile.    Then  for  any  €  >  0,  there  is 

an  n     such  that 

0 

P{B(e)=p  + Jfl,     n>n#  }■*!-*.  (A.l) 

Proof.   As  in  (2.3)  consider, 

#„  (8)  = » ~lEri  (»)  = « "12>«(«,-  -  *,- 8  -  e  (e))  -  p9K  -  e  (ej)  (a.2) 

and  corresponding  directional  derivitives, 

Rn'(b,w)  =  n~lJJr.'(b}w)  =  n~l£[\-9  -\sgn*  (u.  ,x.  w  )]x.  w   (A.3) 
where   sgn*  (u  ,v  )  =  sgn  (u  )    for    u  ^0   and    sgn(v)   otherwise.     We   must 

establish, 


lim  inf        inf        R'   (0,w )  >  0      a. s.  fA4l 

n    -  *  |  to    |    |  =1 

Following  the  approach  used  in  the  proof  of  Theorem  2.1  we  begin  by  show- 
in  £  that  for  fixed  w  , 


12 


R  'n  (0,w  )  -ER'n  (0,w  )-0     a.s.  (A.5) 

Since   |  r '.  (0,tv  )  |   ^    \x.  w      application  of  Kolmogorov's  criterion  yields, 


X 

2  ..„  ,2  ,.2 


EVar(r'.%w))/i    <   E(x.w)/i     <«  (A  6) 

i  =i  »  =i     • 

by  (D2).    This  may  be  strengthened  to  uniform  convergence  on  the  sphere 

\w         =1  using  (D2)  and  the  continuity  of  R  '    (0,w  )  in  w  .    Hence,  we 
have, 

lim  inf        inf       Rn  '(Q,w  )  -  lim  inf        inf       ERn  '(0,w  )        a.s. 

Now, 


n    v    ' 
n    -  x  I  to    I    I  =1  n    -  *  I  to    I    I  =1 


(A.7) 


[F(Q(6))-e]z.  w         if  1. 1»  a  0 
&'(<>,•)  =  ■! '  '  (A.8) 


-[ 


[F(Q  (0)  -  0)-6]x.  w         otherwise 
so  setting  m  (6)  =  min{F  (<?  (6))-6,6-F  (Q  (8)-0)}  we  have, 

Er.'(0,w)  >  m(6)  |  or .  to  |,  (A.9) 

hence,  using  (Dl), 


lim  inf        inf       ERn  '(0,w  )  >  m  {Q)d   >  0, 

n    -  x  |  w    I    I  =1 


(A.  10) 


which  completes  the  proof. 

An  immediate  consequence  of  the  lemma  is  that  under  the  same  condi- 


tions, 


P{QY0)  =  QYW  =  Qr(e+0)  =  Qr(8+0),  n    >  no}>  1  -  X(A.ll) 
which  implies, 


limsupFy(C?r(e))a:Fr(Qy(e)) 

n    -  * 

lim  inf  FY(QY(9-0))  ^  FY(QY(9-0)) 


y\^Y 


y\^Y 
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(A.12) 


(A.13) 


n    -  x 


Which  together  with  (  3.16-17)  imply  that  at  points  of  discontinuity  in  F  we 
have  the  required  convergence.  Since  there  are  only  countably  many  such 
points  this  completes  the  proof. 

Remark.    The  following  example  illustrates  the  necessity  of  the  lemma. 
Let  Q  (9)  =0  for  9€(0,1]  so  that  the  associated  df  is 


Consider, 


and 


F( 


(0    a  <0 
1      a  >0 


QW 


\  n 


6€((V4] 

e*(v] 


(A.14) 


(A.15) 


-1 


F(u)  = 


0 

u  <  -n 

Ml 

-l                -l 
-n     <«  <n 

1 

-l 
n     ^u 

(A.16) 


Then   §()  satisfies  (3.10-11)  but  not  (A.ll).    Q    actually  converges  to  Q 
uniformly,  but 


\\mF(QW)  =F(0)  =  *#  1  =F(0) 


(A.17) 


n  -w 


so  F    fails  to  converge  to  F .  Note  that  since  the  pointwise  limit  of  F    is 


14 

neither  right  or  left  continuous  at  0  the  limit  fails  to  be  a  proper  distribution 
function. 
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